Distributed data processing systems communicate by the exchange of messages. Various products are known which allow messages to be transmitted between heterogeneous platforms, such as IBM's MQSeries products (“IBM” and “MQSeries” are both trade marks of International Business Machines Corporation). Such transmission is not intelligent in that neither the data content nor the internal format of the messages are transformed so as to be intelligible to applications on the receiving platform. This is because the format of these messages may be inherent to particular nodes of the system or the format may be determined by the specific different applications which are running on the system. In both cases, it is necessary to transform message formats originating from one system component (operating system or application) to formats compatible with other system components. Such transformations have long been known in the art, for example, conversion between different date conventions (DD/MM/YY to MM/DD/YYYY, say).
Although such conversion could be performed by the application programs themselves, in one recent development in the message processing art, so-called message brokers have been developed to route messages intelligently between nodes and to transform the message formats as required.
Two recent message broker products from IBM, MQSeries Integrator Versions 1 and 2, both include a so-called “formatter” which transforms messages from one application format into another. The formatter used in both the IBM products is based on one licensed from New Era of Networks, Inc (NEON) and is described in International Patent Application WO 98/30962 “Method for content based dynamic formatting for interoperation of computing and EDI systems”. As one example, these products can transform messages in fixed format, (where each message, and each field within the message, has a specified length in bytes) into standard XML BOD (Business Object Descriptor) messages (XML is the abbreviation for Extended Markup Language).
A general illustration of message transformation is shown in FIG. 1 from which it can be seen that transformation includes both mapping (i.e. relocating) fields in an output message and translation (i.e. expressing values in a different code or convention). An input message 10 originating in a first application consists of four fields 11, named FIELD 1 to FIELD 4. A formatter 12, with access to prestored format definitions and transformation rules for different applications in a database 13, maps FIELDS 1, 2 and 3 to different relative positions in an output message 14, converting them and FIELD 4 as necessary to a different form which will be recognised by a second application. The reformatted output message 14 may then be passed to and processed by the second application.
However, in these systems, the transformations between different types of messages must be predetermined manually and loaded into the transformation engine (formatter and database). This can lead to a very large number of transformations having to be considered as the number of message formats and types of message, even in one system or application, can be very large, particularly if every conceivable transformation must be explicitly recorded. The NEON system reduces this problem somewhat by breaking down messages into basic named canonical (meta-data) components, common to different applications and formats and by using matching of canonical data to determine the appropriate output message conversion. Even this information about transformations at the canonical level, although cutting down the sheer volume of individual transformation pairs to be stored, must still be determined in advance and entered manually. This task is usually performed by a System Administrator using a graphical user interface to enter complete message definitions for different applications and the specific correspondence with format meta-data components into the formatter's database. Typical meta-data could include such terms as “floating point number”, “tag” or “delimiter”.